gh-122622: Add PyREPL simple_interact.statement_submitted_hook by edvilme · Pull Request #157241 · python/cpython

edvilme · 2026-09-09T23:12:34Z

Supersedes #144724

Adds a statement_submitted_hook hook to _pyrepl.simple_interact that gets triggered after the user has submitted a statement (not necessarily a line) but before it is executed.

Rationale

This should be configurable by external tools such as IDEs who may want to perform an action before a statement is executed, such as displaying custom escape sequences for terminal integration, handling custom logging logic, performing analysis, etc.
This hook should allow said tools to perform an action.

Example (Timer)

>>> import _pyrepl
>>> import sys
>>> import time
>>> class PS1:
...     start_time = None
...     def reset_start_time(self):
...         self.start_time = time.perf_counter()
...     def __init__(self):
...         _pyrepl.simple_interact.statement_submitted_hook = lambda x: self.reset_start_time()
...     def __str__(self):
...         end_time = time.perf_counter()
...         time_delta = end_time - self.start_time
...         return f"# Execution took {time_delta}s \n>>> "
...
>>> sys.ps1 = PS1()
>>> 1 + 1 
2
# Execution took 0.0021207999670878053s
>>> [i for i in range(100)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
# Execution took 0.003159299958497286s

Example (VSCode)

VS Code uses custom escape sequences to indicate when a command execution has started and finished. Here, the hook could be used to print an invisible escape sequence to the terminal. (Note that the example uses ### EXEC ### to represent it visually, but escape sequences are invisible)

With error reporting

Issue: Preexec hook for REPL #122622

AI Usage

AI was used to help with merge conflicts

Co-authored-by: Anthony Kim <anthonykim@microsoft.com>

bedevere-app · 2026-09-17T21:35:44Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

edvilme · 2026-09-19T01:20:41Z

I have made the requested changes; please review again

bedevere-app · 2026-09-19T01:20:47Z

Thanks for making the requested changes!

@brettcannon: please review the changes made to this pull request.

Preserve both ANY and MagicMock when merging the interactive console tests, keeping upstream prompt coverage and statement-submitted hook coverage intact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 51fc76fa-b241-49aa-830c-840d2d812b54

brettcannon · 2026-10-29T21:31:09Z

@pablogsal @ambv feel free to review, but if you're too busy I will just plan to merge in about a week.

johnslavik

As this is (primarily) a _pyrepl convenience for VS Code, I'd try to make this PR smaller.

johnslavik · 2026-10-31T00:00:05Z

+    def test_hook_return_value_is_ignored(self):
+        hook = MagicMock(return_value=True)
+
+        _, namespace = self._run_interactive(["x = 1"], hook)
+
+        self.assertEqual(namespace["x"], 1)


I'm not sure there's a reliable way to test that the return value is ignored. I'd remove this case from being tested.

johnslavik · 2026-10-31T00:00:08Z

+            patch.object(simple_interact, "append_history_file"),
+            patch("_pyrepl.readline._setup"),
+            contextlib.redirect_stdout(output),
+            contextlib.redirect_stderr(output),


I wouldn't rely on mocks of internal attributes.

I'd test this with a PTY instead, maybe in test_pyrepl.TestMain?
That ensures the things that need to be displayed to the user are actually displayed to the user.

Example:

cpython/Lib/test/test_pyrepl/test_pyrepl.py

Lines 2482 to 2492 in d948c28

def test_showrefcount(self):

env = os.environ.copy()

env.pop("PYTHON_BASIC_REPL", "")

output, _ = self.run_repl("1\n1+2\nexit()\n", cmdline_args=['-Xshowrefcount'], env=env)

matches = re.findall(r'\[-?\d+ refs, \d+ blocks\]', output)

self.assertEqual(len(matches), 3)

env["PYTHON_BASIC_REPL"] = "1"

output, _ = self.run_repl("1\n1+2\nexit()\n", cmdline_args=['-Xshowrefcount'], env=env)

matches = re.findall(r'\[-?\d+ refs, \d+ blocks\]', output)

self.assertEqual(len(matches), 3)

(This applies to all tests in this PR, I'd try to fit all of them into a few tests in test_pyrepl.TestMain)

A PTY test makes sense here. The unit tests still exercise real statement execution, so I see no need to replace all of them.

One detail: the example needs flush=True, or the execution marker can stay in the buffer until the statement finishes. Our PTY helper uses -u, so it will miss that case.

johnslavik · 2026-10-31T00:00:14Z

+                try:
+                    statement_submitted_hook(statement)
+                except Exception:
+                    console.showtraceback()


It's now easy to misread a traceback from the hook's exception as an exception that comes from running the user code:

>>> from functools import singledispatch Traceback (most recent call last): File "<python-input-7>", line 2, in hook 1/0 ~^~ ZeroDivisionError: division by zero >>>

(I've set the hook in the session, but even if there's internal VSCode paths included in the traceback that are telling this is some hook hackery, I still think this could be improved.)

Additionally, that traceback is displayed before the user code exception traceback.

I'd prefer:

Reporting the full traceback to reader.debug() (a REPL hacker can then uncomment debug() code for verbose error reporting)

Reporting that an "internal" error happened to reader.error() error overlay (something like "exception occurred while running an internal prompt hook")

I agree the traceback needs a clear hook label, but I prefer to keep it visible. reader.debug() currently does nothing, so that loses useful details.

There is a behavior issue with console.showtraceback() here. It replaces sys.last_exc and the other sys.last_* attributes. If the hook fails on each submission, pdb.pm() sees the hook error instead of the previous user exception. I think hook error reports need to preserve that state. This also needs a regression test.

johnslavik · 2026-10-31T00:00:17Z

+# In the REPL:
+#     >>> print("Hello, world!")
+#     \x1b]633;C\x07Hello, world!
+statement_submitted_hook = None


I appreciate the explicit description of the intent!
OTOH, this is a private API for (mainly) the benefit of VS Code. I'd make this shorter to reduce maintenance costs:

I'd remove the example entirely, people hacking the REPL know how to set a module attribute.

The only comment I'd leave is # not called for commands, return value is ignored
I wouldn't pay much attention to exception handling, the default convention is to report all errors somehow. What would be more concerning is if the errors weren't displayed.

We can shorten this and remove the example, but keep the callback contract. External tools need to know when it runs, what it receives, and how errors behave.

“Submitted input” is more accurate than “complete statement”. Syntax errors and blank input reach the hook too. The exception description also needs to distinguish Exception from KeyboardInterrupt and SystemExit.

johnslavik · 2026-10-31T00:09:12Z

 from .readline import _get_reader, multiline_input, append_history_file


+# Called with the complete statement, which may span multiple lines, after the


Oh, one more thing.

This isn't exactly correct. I think a b c is not a "complete statement", no?

>>> import _pyrepl.simple_interact as si >>> si.statement_submitted_hook = lambda x: print("hook called") >>> a b c hook called File "<python-input-19>", line 1 a b c ^ SyntaxError: invalid syntax

Perhaps we could think if there's a better name?

johnslavik · 2026-10-31T00:31:34Z


+            if statement_submitted_hook is not None:
+                try:
+                    statement_submitted_hook(statement)


Nit: I think it would make sense to heat-treat statement with _strip_final_ident before passing it over.

What would you rather receive on the hook in the case below, - or +?

>>> def f(): ... 'there will be some final indent' ... -hook called with "def f():\n 'there will be some final indent'\n " +hook called with "def f():\n 'there will be some final indent'\n"

edvilme and others added 3 commits September 9, 2026 14:34

pythongh-122622: Add PyREPL pre-execution hook

4c18d3b

Co-authored-by: Anthony Kim <anthonykim@microsoft.com>

Add empty pre_execution_hook for feature detection

b988e6f

Update tests to verify no interference

7d9ddbf

edvilme requested review from ambv, lysnikolaou and pablogsal as code owners September 9, 2026 23:12

bedevere-app Bot mentioned this pull request Sep 9, 2026

Preexec hook for REPL #122622

Open

bedevere-app Bot added the awaiting review label Sep 9, 2026

blurb-it Bot and others added 2 commits September 9, 2026 23:15

📜🤖 Added by blurb_it.

ad17396

Update NEWS

0ef602c

brettcannon self-requested a review September 17, 2026 21:15

brettcannon requested changes Sep 17, 2026

View reviewed changes

Comment thread Lib/_pyrepl/__init__.py Resolved

Comment thread Lib/_pyrepl/simple_interact.py Resolved

Comment thread Lib/_pyrepl/simple_interact.py Resolved

Comment thread Lib/_pyrepl/simple_interact.py Resolved

Comment thread Lib/_pyrepl/simple_interact.py Resolved

bedevere-app Bot added awaiting changes and removed awaiting review labels Sep 17, 2026

brettcannon reviewed Sep 17, 2026

View reviewed changes

Comment thread Lib/_pyrepl/simple_interact.py Resolved

edvilme added 2 commits September 18, 2026 13:27

Address feedback

8ede9b2

Address feedback

6d7dfe0

edvilme changed the title ~~gh-122622: Add PyREPL pre_execution_hook~~ gh-122622: Add PyREPL simple_interact.statement_submitted_hook Sep 19, 2026

bedevere-app Bot added awaiting change review and removed awaiting changes labels Sep 19, 2026

bedevere-app Bot requested a review from brettcannon September 19, 2026 01:20

edvilme added 3 commits September 18, 2026 18:29

Merge branch 'main' into pyrepl-hook

d45c1d3

Show tracebakcs

e1186d5

Add tests to make sure hook runs before exception

f615af2

brettcannon added type-bug An unexpected behavior, bug, or error needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Oct 1, 2026

brettcannon added the topic-repl Related to the interactive shell label Oct 1, 2026

Update comment

f766b48

bedevere-app Bot added the type-feature A feature request or enhancement label Oct 13, 2026

edvilme and others added 2 commits October 13, 2026 14:51

Fix incorrect comment

a81c71a

edvilme force-pushed the pyrepl-hook branch from b5f84b1 to a98d37a Compare October 28, 2026 20:57

brettcannon approved these changes Oct 29, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting change review labels Oct 29, 2026

Merge branch 'main' into pyrepl-hook

d948c28

johnslavik requested changes Oct 31, 2026

View reviewed changes

johnslavik reviewed Oct 31, 2026

View reviewed changes

	def test_showrefcount(self):
	env = os.environ.copy()
	env.pop("PYTHON_BASIC_REPL", "")
	output, _ = self.run_repl("1\n1+2\nexit()\n", cmdline_args=['-Xshowrefcount'], env=env)
	matches = re.findall(r'\[-?\d+ refs, \d+ blocks\]', output)
	self.assertEqual(len(matches), 3)

	env["PYTHON_BASIC_REPL"] = "1"
	output, _ = self.run_repl("1\n1+2\nexit()\n", cmdline_args=['-Xshowrefcount'], env=env)
	matches = re.findall(r'\[-?\d+ refs, \d+ blocks\]', output)
	self.assertEqual(len(matches), 3)

		from .readline import _get_reader, multiline_input, append_history_file


		# Called with the complete statement, which may span multiple lines, after the

Conversation

edvilme commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale

Example (Timer)

Example (VSCode)

With error reporting

AI Usage

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-app Bot commented Sep 17, 2026

Uh oh!

Uh oh!

edvilme commented Sep 19, 2026

Uh oh!

bedevere-app Bot commented Sep 19, 2026

Uh oh!

brettcannon commented Oct 29, 2026

Uh oh!

johnslavik left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnslavik Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnslavik Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pablogsal Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

johnslavik Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pablogsal Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

pablogsal Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

johnslavik Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pablogsal Oct 31, 2026

Choose a reason for hiding this comment

Uh oh!

johnslavik Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnslavik Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

edvilme commented Sep 9, 2026 •

edited

Loading

johnslavik left a comment •

edited

Loading

johnslavik Oct 31, 2026 •

edited

Loading

johnslavik Oct 31, 2026 •

edited

Loading

johnslavik Oct 31, 2026 •

edited

Loading

johnslavik Oct 31, 2026 •

edited

Loading

johnslavik Oct 31, 2026 •

edited

Loading

johnslavik Oct 31, 2026 •

edited

Loading